A new family of preconditioned iterative solvers for nonsymmetric linear systems*

A new family of iterative methods, the family of EN-like methods, is introduced, and its relationship to other methods is investigated. The complexity and convergence behavior of the new methods as well as their restarted and truncated versions are examined. The methods are also shown to be suitable in the context of inner/outer iteration schemes. Their adaptive versions are included into a robust software package PARASPAR, and numerical experiments are presented, which demonstrate the efficiency of several members of this new family in comparison with other known methods. I . I n t r o d u c t i o n There still is a great need to find a robust parallel iterative solver and preconditioner for a general sparse linear system. A large number of iterative methods have been developed, which, when convergent, are efficient. Such methods, however, fail often, and the more robust methods available tend to converge too slowly. Many preconditioning techniques have been proposed with various restrictions on their applicability. The more general and robust ones tend to be costly in sequential terms and can have difficulty exploiting more than a moderate number of processors when implemented in parallel. In this paper, we investigate preconditioned iterative solvers based on rank-one updates for the nonsymmetric linear system A x = b where A is a general sparse matrix. Our goal is to design and implement an efficient robust iterative solver for such systems. Specifically, two families of algorithms are considered: (i) the family of Broyden algorithms for nonsymmetric linear systems, (ii) the family of EN-like methods, a new family of methods, which includes a method proposed by Eirola and Nevanlinna [ 12]. * This research was supported in part by the National Science Foundation under Grant No. CCR-9120105. * Corresponding author. E-mail: [email protected]. t E-mail: [email protected]. 0168-9274/95/$09.50 (~) 1995 Elsevier Science B.V. All rights reserved SSDI 0168-9274(95)00088-7 288 U. Me&r Yang, K.A. Gallivan/Applied Numerical Mathematics 19 (1995) 287-317 In the past, methods of family (i) had a bad reputation for solving linear systems, but recent efforts [ 11 ] have shown that different line search principles lead to versions that are competitive with GMRES [ 24]. Under certain assumptions, members of both families will terminate after a finite number of steps and have local superlinear convergence. As with other iterative methods such as GMRES, the full methods are too expensive, and restarted, truncated and adaptive versions must be considered. The computational complexity per iteration step of methods of family (ii) is almost twice as high as the corresponding methods of family (i) and GMRES. Whereas the full EN-like methods in many cases only converge about twice as fast, their restarted versions often converge significantly more than twice as fast as the corresponding Broyden methods and GMRES and are therefore more efficient. We will also see that they often require less memory than the corresponding Broyden methods and GMRES. Iterative methods in both families require, like most other methods, a good preconditioner in order to be robust. There are different ways to precondition iterative methods. We will consider here two different types of preconditioners, the use of an inner iterative method as a preconditioner similar to GMRESR [29] or FGMRES [22] and an incomplete LU factorization with numerical dropping. For the former type of preconditioning the new algorithms are considered as an inner as well as an outer method. The latter preconditioner is taken from PARASPAR, a robust parallel software package based on Y12M [ 15], which has many other interesting features. The new family of methods appears to be very suitable for the strategy used in PARASPAR that gives it its robustness. A more detailed discussion of the results of this paper as well as the proofs for the theorems and lemmas can be found in [19]. 2. Two families of iterative linear solvers 2.1. The fami ly o f Broyden methods An important class of methods based on rank-k updates are the quasi-Newton methods [7]. The purpose of quasi-Newton methods is to determine the zero of a function F or minimize a function G. They approximate the Jacobian of F or the Hessian of G, which is symmetric and often positive definite, or their inverses. There are a variety of effective quasi-Newton methods, such as the Fletcher-Powell-Davidon method and the BFGS method [7]. These methods, however, assume symmetric (and often positive definite) matrices, and we will not consider them here, since our goal is to solve nonsymmetric linear systems. Instead, we will focus on some variants of Broyden's method [ 3], a quasi-Newton method, which is suitable for solving nonsymmetric linear systems. F is defined here by F ( x ) = Ax b, and its Jacobian equals A. In its most general form, Broyden's method is given by Algorithm 1 ( Broyden's method) . Initialization: Xo, Ho arbitrary, ro = b Axo. For k = O, 1 . . . . : Pk = Hkrk qk = Ap~ (1) (2) U. Meier Yang, K.A. Gallivan /Applied Numerical Mathematics 19 (1995) 287-317 2 8 9 Xk+l = Xk + olkpk r k + 1 = r k Olkq k (p~ Hkqt) f ~ Hk+l = H~ + fHqk